Represented in Moving Image

c 1996 S. L. Minneman and S. W. Smoliar ECAI 96 Workshop. Edited by G.Vouros, W.Klas, T.Sellis, M.Vazirgiannis An approach to semantics based on traditional paradigms of knowledge representation (e.g., developing reductionist models of video document genres), while not entirely o the mark, may be signi cantly misdirected. Understanding the semantics of video and multimedia must begin with understanding how video (and lm) are \read" and \written." The purpose of this paper is to set an agenda for coming to an understanding of reading and writing multimedia and to address the representationalist implications of achieving that end. Further, we illustrate this research agenda, showing how we have applied concepts and methods from lm theory (to the reading process) and interaction analysis (to the work of multimedia production), and what our preliminary ndings might mean for computational support and knowledge representation in multimedia.